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This paper analyzes the performance of various predictive quan- 
tizing schemes both for noiseless and noisy channels. The fidelity cri- 
terion used to define optimum performance is that of minimum 
mean-squared error. The first part of this paper compares differential 
pulse code modulation (DPCM) with a system that lacks the feedback 
around the quantizer. Such a system (that is called D*PCM in this 
paper) is actually a pulse code modulation (PCM) system with a pre- 
filter and a postfilter. In the second part of this paper a noise-feedback 
coding structure is used as a framework for a unified analysis of pre- 
dictive quantizing schemes with a frequency-weighted mean-squared 
error as the performance criterion. The last part of this paper extends 
the analysis to include the effects of channel transmission errors on 
the overall performance of these predictive quantizing schemes. It is 
shown that DPCM and D*PCM when appropriately optimized are less 
sensitive to channel errors than PCM, and that the performances of 
DPCM and D*PCM are almost identical in the case of high bit-error 
rates. 

I. INTRODUCTION 

Predictive quantizing schemes employ prediction to exploit the in- 
herent redundancy of input signals; the difference between the actual 
sample of an input signal and its estimate is quantized and transmitted 
to the receiver in a digital format. If the samples of the input are highly 
correlated, then the variance of the samples of the difference signal will 
be significantly less than the variance of the input samples. Hence, the 
overall error between input and output of the communication system 
will be lower than that of a conventional pulse code modulation (PCM) 
system. The first part of this paper compares PCM and two differential 
(predictive) pulse code modulation systems (see Fig. 1). A noise-feedback 
coding structure is then used as a framework of a unified analysis of 
predictive quantizing schemes (including those of Fig. 1) on the basis 
of a frequency-weighted error criterion. The last part of this paper ex- 
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Fig. 1 — Structures of PCM, DPCM, and D'PCM coders. 



tends the analysis to include the effects of channel transmission errors 
on the overall performance of these predictive quantizing schemes. 

The differential pulse code modulation (DPCM) system has a feedback 
around the quantizer resulting in a prediction that is based on previous 
reconstructed samples (Fig. lb). The idea is to base the prediction on 
information that is also available at the receiver. 1 The D*PCM system 
is an open-loop quantization scheme, 2 i.e., previous samples of the input 
are used for predicting the actual input samples (Fig. lc). The term 
D*PCM indicates that we have denoted by d* n the sequence of difference 
samples in the open-loop system. It is important to realize that coder 
and decoder calculate different prediction values. We also see, from Fig. 
lc, that in this latter scheme the prediction network can be replaced with 
a more general linear network. The resulting scheme is then actually a 
PCM scheme with pre- and postfiltering and we shall use this latter 
scheme to derive bounds for the D*PCM performance. 

Let us briefly discuss the differences between DPCM and D*PCM. We 
denote by q n -d n — r n and q n = d* n — r n the quantization errors of 
DPCM and D*PCM, respectively, and by r n the quantized versions of the 
difference samples. 

It is easy to see that the total coding error 



t n = x n - y n 



(1) 
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between encoder input x n and decoder output y n is given by 

t n = q n for DPCM (2) 

provided the channel is error-free and the decoder uses the same pre- 
dictor of frequency response P(co). On the other hand, 

t n = qn + (X n ~ X n ) for D*PCM, (3) 

i.e., the total error is increased by the difference between the prediction 
values x n and x n of the predictors of encoder and decoder, respectively. 
Alternatively, we may write 

t n =q n *h n forD*PCM (4) 

provided that the networks of encoder and decoder are reciprocal. In eq. 
(4), * denotes discrete-time convolution, and h n ; n = 0,1,2 ... is the 
impulse response of the linear decoder network. The result can be easily 
derived by recognizing that 

*n ~ x n y n 

= x n -r n *h n 

= x n - (d* n ~q n )*h n 

= q n *h n +x n ~ x n *g n *h n 

= Qn*h n 

since g n *h n = 5„ and x n *b n = x n (g n is the impulse response of the en- 
coder network and 5 n is the Kronecker delta). 

Quantization errors are approximately white noise samples if the 
number of quantizer levels is sufficiently high. Hence in D*PCM the total 
error is nonwhite noise with each quantization error causing an infinite 
output sequence. This error propagation effect is also sometimes called 
error accumulation; this latter term, however, should be used cautiously, 
because it may seem to imply that D*PCM cannot give any improvement 
in signal-to-noise ratio (SNR) over PCM, or, even more strongly, that the 
overall performance can only be degraded. 

One purpose of this paper is to analyze and explain the differences 
between DPCM and D*PCM. The D*PCM coding system has been analyzed 
by Bodycomb and Haddad 3 ; their approach has been based on a pre- 
dictor optimized for a minimum prediction error variance and they have 
shown that this predictor as a prefilter for a Gaussian process produces 
the same total error variance as a PCM system. We shall use the term 
MMSE predictor to describe such a predictor (that has been optimized 
for a minimum prediction error variance). We shall see very shortly that 
this D*PCM scheme can perform better than PCM provided that the 
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predictor is reoptimized; the MMSE predictor is not optimal in this 
context. 

We have already mentioned that D*PCM can be viewed as a PCM 
scheme with pre- and postfilters. If we model the quantizer in the PCM 
scheme as an additive noise source, the optimization of this scheme is 
almost identical to a joint optimization of pre- and postfilters in com- 
munication systems in which channel errors are present. Many contri- 
butions have been made to this problem both for continuous-time and 
pulse-modulated communication systems. 4-13 One common result that 
can be extracted from these papers is that half-whitening of the input 
spectrum minimizes the overall mean-squared error (MSE) between 
input and output of a communication system with pre- and postfilters 
in which additive white noise (either caused by a quantization or by 
channel errors or by both effects) is present. Half -whitening is obtained 
if the magnitude-squared frequency response is inversely proportional 
to the square-root of the power density spectrum of the input signal. 

PCM schemes with pre- and postfilters do not take into account that 
in systems with quantizers not only can the input spectrum be shaped 
to improve the overall performance but that, additionally, the quanti- 
zation noise spectrum can be shaped as well to further improve the 
performance of the system. This problem has been discussed in detail 
by Kimme and Kuo 14 and later by Brainard and Candy 15 on the basis 
of different coder configurations. These coders have in common a 
feedback of filtered quantization noise to the input of the quantizer and 
we shall use the term noise-feedback coding (NFC) for this approach. 
Our analysis differs from earlier contributions in that a power constraint 
on the quantizer input is not needed. We offer a simplified solution based 
on well-known results in prediction theory. 

It is the aim of this paper to discuss the differences between DPCM and 
D*PCM both for noise-free and noisy channels and to show how they 
relate to noise-feedback coding if the basis of the comparison is a fre- 
quency-weighted minimum mean-squared error. The organization of 
this paper is as follows: in Section II we calculate the differences in 
performance between predictive quantizing schemes with and without 
feedback around the quantizer. We show that D*PCM can perform better 
than PCM for all nonwhite input spectra, but that its performance is al- 
ways below that obtainable with a DPCM scheme. A bound will be derived 
for the differences in performance between these two schemes, and a 
first-order Markov source will be used as an example to explain these 
differences. Section III analyzes noise-feedback coding; its structure is 
given by a prefilter followed by a quantizer with feedback around the 
quantizer. 14 We note that PCM, PCM with noise feedback, DPCM, and 
D*PCM are special cases of this configuration; thus a unified approach 
is possible. We optimize this coder with a frequency- weighted mean- 
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squared error as the performance criterion and show that the prefilter 
has to be a whitening filter for the input signal irrespective of the chosen 
frequency-weighting and that the feedback filter is the MMSE predictor 
of the weighted input spectrum. We also give frequency- weightings for 
which the noise-feedback scheme degenerates to DPCM or to D*PCM. 
Section IV extends the analysis to include the effects of channel trans- 
mission errors on the overall performance of predictive quantizing 
schemes. It is shown that the effects of these errors on total MSE can be 
significantly smaller than those in PCM. In D*PCM the optimum filters 
minimize simultaneously quantization and channel error variances and 
do not depend on the bit-error rate. In the case of DPCM a compromise 
is needed in order to minimize the total effect of both noise sources on 
the total MSE. At high bit-error rates the performances of D*PCM and 
DPCM are almost identical if their prediction networks are identical. 

The analyses are made under certain restrictions; first of all, we use 
the MSE as a performance measure (in Section III a frequency-weighted 
MSE criterion will be taken into account). Second, the analyses are based 
on the assumption that the quantizer can be modeled as an additive 
white noise source. It is known that this model is accurate for quantizers 
with a sufficiently high number of levels. The model is a poor approxi- 
mation, however, in the case of coarse quantization, especially if the 
quantizer input samples are highly correlated. The results of our analysis 
could be extended to these cases by modifying the model but we do not 
consider such an extension in order not to obscure the main results. We 
also assume that the variance of the quantization noise is much smaller 
than that of the signal to be quantized. Comparison with simulation 
results will show the range where our rather restrictive assumptions 
hold. 

II. ANALYSIS OF PCM, DPCM, D'PCM 

The principal aim in this section is to calculate and to compare the 
variance of the total errors in PCM, DPCM, and D*PCM. We assume that 
the input is a sample of a zero-mean stationary random sequence \x n \ 
with autocorrelation function 

R x (k) = E[x n x n+k ], (5) 

power density spectrum (pds) 

S x (u)= £ R x (k)e-*», (6) 

fe=-cc 

and variance 

^ = ^(0) = — f* S x (o))do>. (7) 

2.TT %) —k 
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S x (w) will be assumed to be a rational function of a>. This is not really 
restrictive since most spectra of practical interest can be approximated 
by a rational function. The assumption implies that the pds can be 
represented by S x (w) = t)IA{(x>)A*(u)) where rj x > is a scale factor and 
A(co) is the ratio of two polynomials whose zeros are inside the unit circle 
of the z-domain (factorization theorem for rational spectra). All power 
density spectra are nonnegative continuous functions defined for a> = 
[— 7r,7r]. Since all signals are represented by stationary random sequences, 
all filters are necessarily of discrete-time type. The action of the quan- 
tizer is represented spectrally as white noise added to the quantizer input 
signal: 

S q (u>) = a 2 = R q (0). (8) 

We are interested in the variance 



a? = E[t 2 ] = ±- C'SMdc 

Z7T %J— T 



(9) 



of the total error t n = x n — y n [eq. (1)], where S t (o>) is the power density 
spectrum of the zero-mean random sequence \t n \. A frequency- weighting 
of the total error can easily be added if necessary (see Section III). 

2.1 PCM 

The total error is identical to the quantization error. Hence we 
have 

af=a 2 q = e 2 q .a 2 x . (10) 

The quantizer performance factor e q depends on the properties of the 
quantizer and on the probability density function of the signal being 
quantized; its value is the noise variance generated by a quantization 
of a unit- variance signal. Table IV in Section IV lists various values for 
the cases of 1-bit and 2-bit quantizers. For a given quantizer we thus find 
that 

min\o 2 ]=t 2 q -o 2 x . (11) 

PCM 



2.2 DPCM 

The total error is again identical to the quantization error but its 
variance is now proportional to that of the difference signal: 

cf=a 2 q = e 2 -a 2 d . (12) 

Remark. Some caution is needed if two coding schemes A and B, e.g., 
PCM and DPCM, are being compared. We have to take into account dif- 

1504 THE BELL SYSTEM TECHNICAL JOURNAL, MAY-JUNE 1978 



ferences in the quantizer performance factors e 2 A and e 2 B of the two 
schemes, because the probability density functions of the signals at the 
corresponding quantizer inputs may differ. For example, let A and B 
denote PCM and DPCM, respectively. The gain of DPCM over PCM in 
signal-to-noise ratio is then given as 

4=4 i -4 as* 

The ratio € 2 A /e 2 B is very close to unity if the quantizers have a logarithmic 
characteristic because their performance is relatively independent of 
signal statistics. The ratio is also close to unity if the samples of the coder 
input sequences are Gaussian distributed since all coders discussed in 
this paper employ linear networks which do not affect the Gaussian 
distribution. 

In DPCM systems the prediction is affected by the quantization due 
to the error feedback, i.e., the predictor uses previous reconstruction 
values yj = Xj - tj = xj - qj\ j - n - l,n — 2, . . . instead of the corre- 
sponding input samples xj. The influence of this feedback on the pre- 
diction error variance has been analyzed elsewhere 16 ^ 18 ; it will be briefly 
discussed in the following example but will then be neglected in the 
further analyses. It is known that this simplification is valid if quantizers 
with at least eight levels are employed. If necessary, we shall use the 
terms "real" DPCM and "ideal" DPCM for analyses based on prediction 
with and without error feedback, respectively. 

Example 1: We show the influence of the feedback on the prediction 
error in a DPCM scheme with a first-order predictor of value a. Let p = 
R x {I) hi be the normalized mutual correlation between adjacent sam- 
ples. Without feedback the prediction error variance is 

(14) 

(15) 

for a = a opt = p. The DPCM difference signal is d n = x n — ax n -\ + aq n -\. 
Its variance is given as 





a 2 d =(l + a 2 -2ap)-a 2 x 


with a minimum 






min(4} = (1 - p 2 ) • a x 



__ 1 — 2ap + a 2 
l-e 2 a 2 



on the assumption of a vanishing correlation between input and quan- 
tization error, 16 and its minimum variance is obtained with 19 : 



1+'" 2 



£ 

&opt — 



2p 



Mw(rfe) 2 ]. (") 



PREDICTIVE QUANTIZING SCHEMES 1505 



40 



30 



20 



S = 4 



S = 2 




THEORETICAL RESULTS 

O O O SIMULATION RESULTS 



0.6 



0.7 



0.8 



0.9 



1.0 



Fig. 2 — DPCM performance: SNR vs. value a of the predictor coefficient. Source: Gaussian 
first-order Markov source with correlation p = 0.9625. Quantizer: B-bit quantizer optimized 
for Gaussian signals. Predictor: Previous sample prediction with coefficient a. 



The value a opt is not critical, however. We use a opt 
total MMSE: 



min{of} = cj 



-P 2 



1 - 



*" 



p to determine the 



(18) 



Due to feedback there is an increase in variance by a factor (1 — ejp 2 ) -1 . 
Figure 2 shows the dependence of the signal-to-noise ratio (SNR) on the 
value of the predictor coefficient for various B-bit quantizers and com- 
pares theoretical results obtained from eq. (18) with simulation results. 
It is seen that these are useful approximations in the vicinity of the op- 
timum setting of the predictor coefficient (this is the region where the 
difference signal is almost white noise, and the simulations reveal that 
the assumption of a vanishing correlation between quantization error 
and input signal holds in this case). 

In an "ideal" Nth order DPCM system the prediction of an input 
sample x n is based on previous input samples x n -j\j = 1,2 . . . JSl. Hence 
the prediction scheme is essentially that of D*PCM (see Fig. lc), but it 
is important to realize that these schemes are optimal for different fre- 
quency responses P(a>) of the predictor as will be seen later in this paper. 
Equation (12) shows that the optimum predictor in DPCM is the MMSE 
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predictor. The prediction error variance is given as 

a 2 d =^~ C W SA*)\l-PM\*dv (19) 

and its minimum is reached if the random sequence \d n \ is a white noise 
sequence of variance rj x (full-whitening). Thus we have 

minlirj} = vl (20) 

where tj x is the minimum prediction error variance to be obtained from 
the random sequence \x n \ by passing its samples through a prediction 
error filter with frequency response 1 — P op t (oj) such that 

S x (u)\l ~ Po P t(u)\ 2 = vl (21) 

We note that this minimum can be obtained for any stationary random 
sequence if the predictor impulse response is of semi-infinite length, and 
an iVth order predictor can be employed if the random sequence is 
Markovian of order N. For a given pds S x (a>) Kolmogoroff 20 has given 
the minimum of the prediction error variance as 

V 2 X = mini*}} = exp [^ £ logA («)<*«]. (22) 

This variance rj x is a positive quantity if the process is undetermined, 
i.e., if S x (co) is zero at most at a countable set of frequencies. Otherwise 
the signal is perfectly predictable and the prediction error variance is 
zero then. The normalized prediction error variance 



exp — ( log c S x (o>)du> 



y 2 x = - 9 = - t" " ' (23) 



a 



- f 

2tt J- 



S x (<i))da) 



is called spectral flatness measure 21 and its inverse is the optimally ob- 
tainable prediction gain. The spectral flatness measure can be inter- 
preted as the ratio of the geometric mean of the pds S x (u) to its arith- 
metic mean. It is easy to show 21 that 

< t! < 1. (24) 

From eq. (12) we finally find that 

min {<r 2 } = e 2 • t, 2 , (25) 

DPCM 

2.3 D'PCM 

The quantization error with pds S q (to) = a\ is filtered by the linear 
decoder network with frequency response H{w) = G -1 (co) (see Fig. lc). 
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Thus the total error variance is 

rf-*J-^£|H(«)|V* (26) 

i.e., the quantization error variance is increased by the power transfer 
factor 

a = — f* \H(a)\*da. (27) 

2x %j — ir 

The variance of the quantization error depends on the variance of the 
difference signal d* n : 



.2 — ,2 



«*-«J;r f*<s x («)|G(«)|2deo. 



°i-*i'°h = <i— \ S x («) G(«) 2 d«. (28) 



Thus the total error variance is 

<Tf=e*-a-a%., (29) 

i.e., the total error variance of D*PCM is a times that of an equivalent 
DPCM coding scheme (with <j\ = a^' in the case of fine quantizing and 
identical prediction filters). The impulse response of the postfilter is a 
sequence {l,/ii,/i2, • • •}; thus we have 

a = 1 + £,h$>l (30) 

k=i 

and we find that DPCM outperforms D*PCM for any choice of the pre- 
dictor network. 22 This result also implies that the optimum performance 
of DPCM is better than the optimum performance of D*PCM. This fact 
will be shown very shortly in a slightly broader context. 

Example 2: We calculate now the total MSE of a simple D*PCM scheme 
which employs a first-order predictor of value a. The prediction error 
variance has been given in eq. (14). The power transfer factor of the 
decoder network is 

a = 1 + a 2 + a 4 + • • • = (1 - a 2 )" 1 . (31) 

Thus the total MSE is 

a f =6 2 2 t = e2 l-2a P + a 2 m 

1 — a z 

Note that MMSE prediction (a = p) results in the same MSE as that of 
standard PCM, 3 and that of -* °° for any p if a -*- 1. The minimum total 
MSE is obtained for 



aopt = -a-VT^), (33) 

p 
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Fig. 3 — D'PCM performance: SNR vs. value a of the predictor coefficient. Source and 
coder same as in Fig. 2. 



and its value is 



min|(r?} = €jvT^2-^. 



(34) 



Thus it is possible to reduce the total noise variance by a factor Vl — p 2 
instead of (1 — p 2 ) as in ideal DPCM (with negligible feedback). It is in- 
teresting to note that this is the same reduction that we can obtain with 
a block quantization scheme of blocklength 2 (this is a scheme where the 
sum and the difference of adjacent samples are quantized indepen- 
dently). 23 Figure 3 demonstrates this fact that the SNR is lower than that 
obtainable with DPCM (Fig. 2). In the case of quantizing with one and 
two bits/sample there is, as expected, a significant difference between 
theory and measurements especially for a -values which are not close to 
the correlation coefficient p. This difference is now explained by the fact 
that, in the case of coarse quantization, the white noise assumption [eq. 
(8)] does not hold for correlated quantizer input samples. Simulations 
have revealed that higher signal-to-noise ratios can be obtained by 
choosing a decoder coefficient that is higher than that of the coder. Note 
also, from Fig. 3, that no gain over PCM is obtainable for the specific case 
a= p. 

We come back now to the total MSE of D*PCM. Equation (29) is given 
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explicitly as 

of = e 2 [ £ £* S* (co) 1 1 - P(«) 1 2 dcol 

x [^XI' 1 " p(a,) '" 2dco ] (35) 

where we have used our assumptions G(co) = 1 — P(co) and //(co) = 
G -1 (o>) (both assumptions will be reviewed in the next section). Note 
that the term in the first brackets reduces to tj x if P(co) is the MMSE 
predictor [full-whitening, see eq. (21)]. The difference signal d* n is white 
noise then, and, since the last term in eq. (35) is the power transfer factor 
for white noise inputs, it equals a\la\* = fffAf* Therefore the total MSE 
is of = e 2 • <r 2 , and thus equals that of PCM. 3 Note that this statement 
is based on assumptions given in the remark in Section 2.2. We relate 
now the minimum total MSE obtainable with D*PCM to that of DPCM. 
By applying Schwarz's inequality* to eq. (35) we obtain 

min \c 2 \>e 2 q \j- C* VSM d»Y = <%<& (36) 

D'PCM \_2tZ J-* J 

where al/j defines the variance of a process with pds VS X (co) . 
Equality is obtained if 



1 1 - P(co) | 2 VSM = const. (37) 

We conclude that the squared-magnitude frequency response of the 
D*PCM encoder has to be inversely proportional to the square root of the 
pds of the input (half -whitening). Therefore the optimum frequency 
response 1 — P(co) is a MMSE prediction error filter for a pds VS X (co) . 
The corresponding minimum tj 2 /^ of the prediction error variance would 
be just the square root of that to be obtained from an optimum prediction 
of a process with pds S^co) [this can be verified from eq. (22)]: 

|1 - P(co)|2 VSAu) = V 2 vi=Vx (38) 

However, it is important to realize at this point that it is not certain that 
the equality in eq. (37) can be obtained at all in practice. Indeed, equality 
can only be expected if VS X (co) happens to be a rational spectrum, since 
such a spectrum can always be modeled by white noise passed through 
a purely recursive linear filter with poles inside the unit circle. It is clear 
that such a process can be whitened by a one-step ahead prediction error 
filter 1 - P(co). In general, however, the pds VS X (u)) is not rational, and 
the minimum total error variance to be obtained by D*PCM coding is 



t Si fjMdx • // Mx)dx > Ui fi(x)f 2 (x)dx] 2 with equality if /?(* )//!(*) = const, for any 
square-integrable functions A(-) and fs(*)- 
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greater than the right-hand term of eq. (36). Example 2 has shown that 
an improvement over PCM can be obtained with D*PCM. By applying 
Schwarz's inequality to the right-hand term of eq. (36) we have 

«?[r- f*V^RdJ 2 <eJa|=min{(r?} (39) 

I2ir J-* J pcm 

with equality only if \x n \ is a white noise sequence, i.e., an improvement 
over PCM can be obtained for all nonwhite processes. Equation (30) has 
already indicated that DPCM outperforms D*PCM for any nonwhite pds 
S x (a>). We can make this statement more quantitative by comparing the 
total error variances of D*PCM (in the most favorable case, given if the 
pds VS X (u>) is rational) and that of DPCM. To be more specific, we use 
eqs. (22)-(25) and (36), and find 



min {<rfj 

DPCM 



exp — J log c S x (co)dco 

^ 

min \c 2 t \ \±- CVSAa)du>Y 

D'PCM L27T J-ir J 




< 1. (40) 



We conclude that the ratio of the total error variances of DPCM and 
D*PCM is upperbounded by the squared spectral flatness measure y 2 /^ 
of the pds VS X (u>) . The validity of the right-hand inequality in eq. (40) 
has already been stated in eq. (24). Equality is obtained if and only if \x n \ 
is a white noise sequence. In other words, DPCM outperforms D*PCM for 
all nonwhite spectra. 

Example 3: Example 1 has shown that the total MSE of a DPCM scheme 
with a first-order predictor is smaller by a factor /3 2 = 1 — p 2 than that 
of PCM (provided that the quantizer performance factors are identical 
in both cases). In D*PCM, the corresponding factor is /3 2 = Vl — p 2 (see 
example 2). The value p is in both examples the one-lag normalized 
autocorrelation coefficient of an otherwise arbitrary (though stationary) 
random input sequence. Let us assume now that this sequence is a 
first-order Markovian sequence with autocorrelation 

«*(*) = p 1*1. <rj (41) 

and pds 

s * (w) = 71 l~f * < 42 > 

1 + p z — 2p cos a> 
PREDICTIVE QUANTIZING SCHEMES 1511 



The best DPCM performance is still obtained with a first-order predictor 
since [x n ] is Markovian. The D*PCM reduction factor (3 2 = Vl — p 2 is not 
optimal, however. It is interesting to compare it with the upper bound 
to be obtained without the constraint of the readability of the prefil- 
ter. 

Using eqs. (36) and (42) we find 

min {a?}>€?.02.oj! (43) 

D*PCM 

where 

2 ^1-P 2 )(*) 2 **(f.p). (44) 

F(7r/2, p) is the complete elliptical integral of the first kind: 



(I-)- j: 



Vl — p 2 sin 2 

We conclude that a D*PCM coding of a first-order Markov source is up- 
perbounded by a reduction factor 

2 = (I -p 2 )\l+\p 2 + ^-p 4 + •••]' (46) 



instead of (1 — p 2 ) for DPCM and Vl — p 2 for the one-tap D*PCM. For 
p = 0.9625 the one-tap D*PCM coding results in a total MSE that is by a 
factor 3.7 greater than that of (ideal) DPCM, and it is lower-bounded by 
a factor 3.0 (for p = 0.85 the corresponding values are 1.9 and 1.8, re- 
spectively). 

The foregoing discussion has shown that D*PCM is a suboptimal coding 
scheme if the performance criterion is the unweighted total error vari- 
ance. The next section will demonstrate that D*PCM is an optimum 
coding scheme for a specific frequency-weighted error criterion, and 
Section IV will show that its performance bounds the overall perfor- 
mance of DPCM coders in the presence of channel errors. As a final re- 
mark we mention that D*PCM based on adaptive prediction with a sep- 
arate transmission of the predictor coefficients has been used recently 
for speech coding purposes 24 to improve the subjective performance. 

III. NOISE-FEEDBACK CODING 

In the preceding chapter we have optimized pre- and postfilters for 
a quantizing scheme (D*PCM) and have shown that DPCM has a superior 
performance for all nonwhite processes. At a first glance this seems to 
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Fig. 4 — Noise-feedback coding scheme. 

be surprising since D*PCM coding is also based on linear filtering. In 
the D*PCM analysis, however, we did not take into account that not only 
the pre- and postfllters are under our control, but also the quantizer. To 
be more specific, we may additionally filter the quantization noise 
whereas in typical pre- and postfiltering applications it is the channel 
that is perturbed by noise; that noise is not under control of the designer. 
Figure 4 demonstrates how the filtering of the quantization noise is ob- 
tained. The quantization noise, i.e., the difference between input and 
output of the quantizer, is fed back through a linear filter with frequency 
response Q(a>) and is added to the input. Q(o>) is required to have a 
minimum delay of one sampling time for stability reasons. The purpose 
of the feedback scheme is a reshaping of the spectrum of the quantization 
noise such that the total error variance is minimum. It should also be 
mentioned at this point that there exists also another linearly equivalent 
scheme, the direct-feedback coder, which has been studied by Brainard 
and Candy. 15 It uses a prefilter of frequency response A(a>), and the 
output of the quantizer (not the quantization noise!) is first passed 
through a feedback filter of frequency response B(a>) and then added 
to the prefiltered signal to form the quantizer input. The equivalence 
to the noise-feedback coder is given by G(u>) = A(co)/(l — S(co)) and 
1 - Q(o>) = 1/(1 - B(w)). 

3. 1 Derivation of the basic formula 

Let us, as before, represent the quantizer as a device that adds sig- 
nal-independent white noise of pds S q (co) = o- 2 to the signal. Due to this 
assumption we can also replace the feedback-quantizer with a nonwhite 
noise source of pds 



S B («)-S 9 (u)|l-Q<«)| 2 



= <x 2 |l-QM 



(47) 



The feedback acts as linear filter on the open-loop quantizing noise, and 
the effective quantization noise is colored noise then. Additionally we 
may introduce a subjective noise-weighting function S w (u)) whose inverse 
describes the sensitivity of the sink to uncorrelated noise. A small value 
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of S w (a>) indicates that a high error variance is acceptable in that specific 
frequency region and vice versa. It is well known that such a fre- 
quency-weighting can only serve as a first approximation of noise sen- 
sitivity since it does not take into account nonlinear effects and depen- 
dencies on local properties of the signals. Nevertheless, the frequency- 
weighted noise power is a quite popular design criterion partly due to 
the lack of better distortion measures. What is worth emphasizing is that 
S w (u) can be interpreted as the squared-magnitude frequency response 
of a noise-weighting filter. Therefore it is reasonable to assume that 
S w ((»)) is a rational function of to. S w ((a) can also be explained as the 
output pds of such a weighting filter whose input is a white noise process. 
This interpretation suggests to define a variance 



«* -— c 



S w (u)da, (48) 

a minimum prediction error variance 

V 2 W = exp [^ §* ]ogA(«)<to], (49) 

and a spectral flatness measure 

l\ = vl/ol (50) 

in accordance with eqs. (7), (22), and (23). 
The MSE of a NFC coder is given by 

of = e 2 q [^ £S«(»)|G(«)|W] 

• [^ f^S w (o>)\H(o>)\ 2 \l - Q(a>)| 2 dcol (51) 

where we have used eqs. (26), (28), and (47). Note that the MSE does not 
include a linear distortion term resulting from a possible mismatch be- 
tween prefilter and postfilter. We are now free to choose linear filters 
G(w), H(u), and Q(o>) such that the frequency-weighted error variance 
is minimized. This is obtained by a proper preshaping of the signal 
spectrum and the quantization noise prior to transmission. The general 
design of this NFC scheme has been studied by Kimme and Kuo in the 
context of picture coding. 14 Our objective is to show the connection of 
this noise-feedback scheme with those discussed so far and to show how 
subjective noise-weighting functions influence the design. Our approach 
will also be different from that in Ref. 14 since a constraint is not needed 
in the optimization procedure. The MMSE design problem is to find the 
optimum combination of prefilter G(cu), postfilter H(to), and feedback 
filter Q(a>) for a given pds S x (<a) of the input signal, and a weighting 

1514 THE BELL SYSTEM TECHNICAL JOURNAL, MAY-JUNE 1978 



function S w (w). Let us first assume that G(a>) and Q(a>) are given and 
let us determine H(io). In an optimized system the total error x n — y n 
must be orthogonal to the output of the postfilter and thus orthogonal 
to the data r n used in that filter (the sequence \r n \ is the decoder input 
sequence). It can be shown that this condition also holds in the case of 
weighted total errors. If the postfilter H (a>) is not constrained to be 
physically realizable (its characteristics can always be approximated 
arbitrarily closely by allowing for a sufficient time delay), the optimum 
filter is given by the Wiener-Hopf condition 

tf„p,M=!ff (52) 

o r (w) 

where S rx (a>) = G*(a>)«S x (a>) is the cross-spectrum between the sequences 
\r n ] and \x n \. Thus we see that 



Hopt («) = ,„, xlo „ , //„„.. ^TTT^- (53) 



G*(w)-S x (<») 
|G(<0)|%(«) + S f («)|l-Q(«)|> 

We shall restrict our attention to the case of a small quantization noise 
variance: 

|G(a,)| 2 S x (a,) 

Sq(u)«ii n , s.o for a« to. (54) 

|1 - Q(a>)| 2 

Equation (53) becomes 

H opt (u>)=-^-, (55) 

G(w) 

so that G(co) and //(oj) are reciprocal filters for any given G(o>) which does 
not violate the assumption of eq. (54). It is seen that choosing reciprocal 
coding and decoding filters is not just a convenience but a requirement 
by the MSE criterion. By applying Schwarz's inequality to eq. (51), we 
find 

min|a 2 ) = € 2 r^j; £ VS x (co)S U) (o;)|l - Q(co)| 2 duT (56) 

for any given S w (co) and Q(«). This minimum is reached if 



\a*M\* = c* ^-Ml 1 - QMI 2 ( 57) 

S x (co) 

where C is a constant. Equations (51) and (56) can be used to calculate 
the error variances of various coding schemes. 

3.2 Optimization of the noise-feedback coder 

We shall now derive two conditions which have to be met by Q(co) and 
G op t(<a), respectively. We apply again Schwarz's inequality to eq. (56) 
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and thus have 

min{<r 2 } < e\ ~ C S x (a>)S„ (a,) 1 1 - Q(o>)\*du> (58) 

with equality if the integrand is a constant. Note that the term 1 — Q(w) 
describes a prediction error structure. Therefore equality is obtained 
by choosing Q(a>) to be the MMSE predictor of a random sequence of pds 
S x (a))-S w (u) and we have [in the notation of eq. (21)]: 

S Au)S w (to)\l - Q opt (u>)\ 2 = vL = nl • vl (59) 

where Q opt (o>) is the MMSE predictor that whitens a random sequence 
of pds S x (o))'Su;(a>), and where rjxvl is its MMSE. The right-hand equality 
in eq. (59) can be obtained from Kolmogoroff s result [eq. (22)] by sub- 
stituting S x (u) with its frequency- weighted version S x (u))S w (u). When 
comparing eqs. (56) and (59) we find 

min{<r 2 } = ef- V 2 - V 2 . (60) 

NFC 

We conclude that the frequency-weighted total error variance of an NFC 
scheme with given quantizer is determined by the product of the pre- 
diction error variances of the spectra S x (u) and S w (o)), and that Q op t(u) 
is the optimal predictor of a pds S x (u)-S w (u)). We also have, from eqs. 
(57) and (59), that S x (o>) | G opt (w) | 2 = const. This implies, however, that 
I Gopt (<o) | 2 is a filter which whitens S x (a>) which has been assumed to be 
rational. Thus a sufficient condition for an optimum NFC scheme is to 
choose as a prefilter an MMSE optimized prediction error filter: 

G opt (<») = 1 - PopM (61) 

where P opt (co) is defined by 

S x (co)|l - P opt (a)\ 2 = vl (62) 

Note the important fact that the overall performance is optimized if the 
prefilter is an MMSE prediction-error filter that whitens the input process 
of pds S x (o>) and that this result holds for any choice of the weighting 
function. It turns out that we have to modify the quantization noise 
feedback loop but not the prefilter if a weighting of the noise has to be 
taken into account. We finally find from eqs. (20), (47), (59), and (62) 
that the weighted error spectrum S f (oj) = egO-jS w (u))\l — Q op t(u)\ 2 
• 1 1 — P opt (co) | ~ 2 is constant and thus equals the total error variance [see 
eq. (60)]. We finally note that these optimization results can also be 
extracted from the Kim me and Kuo paper. 14 Musmann has recently 
derived equivalent results based on a more information-theoretical 
analysis. 25 
We shall now briefly discuss two special cases. 
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Fig. 5 — Model of NFC coder with prediction error filter as input. 

Special case: S w (u) = 1 for all u> (DPCM). A comparison of eqs. (59) and 
(62) reveals that Q op t(u) = P pt(u), i-e., the predictor prefilter and the 
feedback filter have to have identical frequency responses. It is easy to 
see 26 that these filters can then be combined to yield the DPCM structure 
of Fig. lb. A DPCM structure is indeed defined by having Q(oj) = P(co) 
and Fig. 5, which is a model of a noise-feedback coder, reveals that no 
frequency-weighting of the quantization noise is possible in this case for 
any choice of the predictor (provided that pre- and postfilters are re- 
ciprocals), since the quantization noise passes both the feedback filter 
with frequency response 1 — P(o>) and its inverse, the postfilter. 

Special case: S w (w) « Sj 1 (w) for all o> (d*PCM). A weighting function 
that is in some sense inverse to the pds of the signal is of importance for 
quantizing acoustic signals since it may avoid a masking of weak signal 
energies in specific frequency ranges by the quantization noise. 

For S w (a) oc S^ico) we find from eq. (59) that 1 1 - Q op t(u)\ 2 has to 
be a constant. This clearly means that Q opt (oj) = for all co; i.e., the best 
coding scheme is now D*PCM. The prefilter is, of course, as for all optimal 
NFC coders, a whitening filter. The D*PCM scheme is suboptimum if it 
is used in connection with other weighting functions. The special case 
of S w (o>) = 1 has been discussed in detail in Section II; we have seen there 
that for S w (u>) = 1 and a MMSE predictor P(co) no reduction in total error 
variance over PCM is obtainable. The above discussion reveals that the 
same coder is, however, optimal for the specific noise-weighting function 
S w ((x>) cc S~ l (io). Indeed, subjective gains of about 6-10 dB have been 
reported for speech signals, for this choice of the prefilter. 26 - 27 

Table I lists various NFC configurations. The performance of some 
suboptimal coders will be compared with the optimal NFC scheme in the 
next part of this section. An elementary example will suggest the manner 
in which the NFC design influences the total error variance. 

Example 4: Assume a noise-feedback coder with just one tap, i.e., Q(w) 
= q«exp(— jw), and a prefilter with the structure of a one-tap predic- 
tion-error filter: G(a>) = 1 - a-exp(-;'<o) (see Fig. 6). Further assume a 
sequence with an adjacent-sample correlation p. We have H(co) = G -1 (w) 
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Table I — Performance comparison of NFC coder configurations 



Noise weighting 



NFC 



DPCM 
Q(co) = P(co) 



D*PCM 
Q(w) = 



Su>(w) = 1 
(no weighting) 

S w («)«S," l («) 



All other 
cases: 



= DPCM 



= D'PCM 



Optimal 
coder 



Optimal coder: 

ll-P^ceSj 1 

Suboptimal 
coder 

Suboptimal 
coder 



Suboptimal coder (upper 
bound half-whitening): 
fl-P| 2 ccSj 1/2 
Optimal coder 
(full-whitening): 
JG| 2 = Jl-P^ecSj 1 
Suboptimal coder 



and the unweighted total MSE [Su,(co) = 1] can be derived from eq. (51); 
we shall omit the intermediate steps. The final result is 



« 9 1 + a 2 — 2ap ._ „ n . 
<r t 2 = A : T^ (1 + Q 2 ~ 2aq) 

I — ft « 



a < 1; q arbitrary. 



(63) 



For a = and q = we have the PCM result of eq. (10). For a = and fi- 
nite q we obtain of = ej)(l + q 2 )-o 2 Xi i.e., noise feedback without prefil- 
tering increases the MSE by a factor 1 + q 2 over that of PCM. For q = 
and finite a < 1 we have the case of D*PCM, i.e., prefiltering followed by 
quantization [eq. (32)]. The best choice of q is q = a if a is given; the 
scheme reduces then to DPCM [eq. (1)] and reaches its MMSE for a = q 
= p. The different cases-have been tabulated in Table II. 

3.3 Suboptimal coding schemes 

The last section has shown that noise-feedback coding is a scheme that 
allows for the optimal shaping of the spectra of the input signal and the 
quantization noise such that the noise-weighted overall error variance 
is minimized. The prefilter is in all cases a MMSE prediction error filter 
1 — P opt (to) which performs a decorrelation of the input signal. The op- 
timal scheme reduces to DPCM (Q op t(co) = P pt (<*>)) in the case of un- 
weighted noise (S w (co) = 1), and to D*PCM (Q op *(co) = 0) in the case of 
S w (u)) cc SgHu). In all other cases, only the general NFC scheme (with 
Q(co) ^ P(co)) is optimal. 



*nO 




OYn 



Fig. 6 — One-tap NFC coder of Example 4. 
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Table II — Error variances of coding schemes with one predictor 
coefficient a opt and one feedback coefficient q opt 





min)ff?| 


Qopt 


Oopt 


PCM 
Noise-feedback PCM 


#1 + A* 



^0 



p 

Qopt 







D'PCM 

DPCM (ideal) 

DPCM (real, i.e., with 
coarse quantization) 


«5(l-p 2 )«r* 

2 1 ~ p 2 j 


-d-vi- P 2) 
p 

<p [seeeq. (17)] 



This section compares the performances of various suboptimal coding 
schemes which belong to the class of NFC schemes and which are derived 
by choosing suboptimum prefilters and feedback filters. The corre- 
sponding error variances can be derived directly from eq. (51). We shall 
omit the intermediate steps in the calculations and optimizations of <r 2 
and shall present the final results only. 

PCM. PCM results if G(u>) = H(«) = 1 and Q(w) = 0. We then have 

min (*?) = $<xl<xl (64) 

PCM 

The quantity o 2 w as defined in eq. (48) represents the subjective gain if 
white quantization noise is weighted. 

Noise-Feedback PCM. We have G(a>) = H(u>) = 1, and Q np t(w) is the 
MMSE predictor of the weighting function S w {o)). Hence we have 

min \a'f\ = t 2 q a 2 x T]l. (65) 

noise-feedback 
PCM 

This result shows that Vw as defined in eq. (49) represents the error re- 
duction obtainable if the quantization noise is optimally shaped. Note 
that optimal noise-shaping reduces the weighted error variance in 
relation to PCM by a factor y 2 w = vU^w which is the spectral flatness 
measure of the weighting function S u ,(co). 

Remark: We have to mention at this point that eq. (65) only holds if y 2 , 
is not close to zero. This implies that the predictibility of S w (u) (which 
is bounded by the dynamic range of the spectrum 28 ) should not be too 
high. Otherwise one could obviously achieve large reductions in error 
(including unbounded ones for weighting functions which are zero over 
a finite segment of the frequency axis; see eq. (22) and the following 
discussion thereof). The noise reductions are obtained by shifting the 
noise in frequency to a range where it is less heavily weighted by S w (co). 
However, the noise-shaping increases the total variance of the quantizer 
input signal since the correlated quantization noise is added to the input 
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signal. Therefore the quantizer step sizes have to be readjusted ac- 
cordingly. As a consequence we obtain an increase in the quantizer 
performance factor e 2 and subsequently an increase in total error vari- 
ance. Spang and Schultheiss 29 have discussed in detail the stability 
problems involved. They have been interested in noise reductions in 
oversampled systems where the noise can be shifted into the high-fre- 
quency range and subsequently eliminated by lowpass filtering. In that 
application we have a weighting function that is zero over a finite seg- 
ment of the frequency axis. As a means for reducing the stability prob- 
lems Spang and Schultheiss propose to keep the number of feedback 
elements R finite. In this case the total error variance is given as 



1 /»*- 
2?r J-,r 



<Jt ~ — I S w ((o) 



1 " E q k e-i k " 



da. (66) 



The feedback network can be viewed as an Rth order predictor. Its op- 
timal coefficients qk\ k = 1,2, . . ,R can be derived from the set of R 
normal equations whose coefficients are just the Fourier coefficients of 
S w (a).™ 

D*PCM. We have G(w) - 1 - P(w) and Q(a>) = 0. 

(i) Lower bound (half -whitening). The lower bound is reached if 
|1 - P 0P M\ 2 = VS w (u)/SAo>). We obtain 

min {tr 2 } = € 2 <x^ (67) 

D*PCM 

where 

/ 

Am = j£ f ' Ai^-SJo,) da,. (68) 

(ii) Full- whitening. Let P(w) be the MMSE predictor for S x (w). It 
follows that 



at = tfcL (69) 



where 



1 /•» 
alw-— I S x (co) • S w (u))dw. 
Air %) —it 



(70) 



DPCM. Let P(u) = Q(a>) be the MMSE predictor for S x (a>). It follows 
that 

min \af\ = e 2 qV 2 x <rl (71) 

DPCM 

Note that optimal prediction has reduced the weighted error variance 
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in relation to PCM by a factor y x = Vx/^x, i-e., by the spectral flatness 
measure of S x (u>). 

Pre filtered DPCM. Let Q(a>) be the MMSE predictor for S x (w). DPCM 
results if P(a>) = Q(co). Prefiltered DPCM results if P(<o) ^ Q(co). By 
varying P(a>) we obtain 

min {of } = efrf <r*^ (72) 

Prefiltered 
DPCM 

where 

ffVF=— f'v/S^Rdco. (73) 

Z7T J —ir 

From Schwarz's inequality we have 0^ ^ o"L Hence we find the result 
that the performance of DPCM can be improved by employing an addi- 
tional prefilter. It is worth emphasizing, however, that we have set Q(a>) 
to be the MMSE predictor for S x (co). The optimal NFC scheme results if 
we are also free to optimize this feedback filter (see below). 

NFC. The minimum total error variance of NFC schemes has already 
been given in Section 3.2 and is repeated here for completeness: 

min (<x 2 } = e 2 q r, 2 x r,l. (60) 

NFC 

Note that optimal prediction and optimal noise shaping properties are 
provided by NFC schemes. Accordingly the total error variance is reduced 
in relation to PCM by a factor y\ • yl, which is the product of the spectral 
flatness measures of input spectrum and weighting function. We shall 
sec very shortly that this scheme is very close to theoretical bounds in 
the case of Gaussian input sequences. 

Example 5: We shall now evaluate the above derived results for the 
specific example of a first-order Markov source of variance tr 2 . and nor- 
malized adjacent-sample correlation p ^ whose power density spectrum 
has already been given in Example 3. Such a source is a useful first ap- 
proximation for modelling the statistics of speech (with p = 0.85 in this 
example) and of television signals (with p = 0.9625 in this example). 

Speech signals. We assume a weighting function which is inversely 
proportional to the pds of the signal. We have already mentioned in 
Section 3.2 that subjective gains of about 6-10 dB have been reported 
for this specific weighting. The foregoing analysis has also revealed that 
the optimal scheme, i.e. NFC is identical to D*PCM. We have 

S.(»)-c*.Sr l (») (74) 

where c 2 can easily be determined if S w (u)) is normalized such that 

max|S u ,(a.)|= 1 (75) 
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Table III — Comparison of coder design parameters 

S w {w) = 1 S w (w) <x SJ'M 

ll (1-P 2 )-o? 

i 2 1 



<r* 



&(£..) 



(1-P 2 )-* 2 


1 


(1 + p) 2 


1 + P 2 


(1 + P) 2 


-2 „2 


1 + P 


^(f.») 



and if, in addition, use is made of the following result: 

min{S x (co)} = ^<r2. (76) 

Table III lists the important quantities for S w (o)) = 1 and S w (u) « 
S^iw). F(-,-) is the normal elliptic integral of the first kind [see eq. (45)], 
and .E(v) is of the second kind. We have 

E (i'*KH* 2 -^ 4 --)- ™ 

The quantity k is given by 

k = VTp/iX + p) (78) 

and E(ir/2, k) is close to unity for p > 0.85. 

From Table III we find that the total error variance can be reduced 
by a factor y 2 = Vx/^x = 1 — p 2 by means of optimal prediction and ad- 
ditionally by a factor y* = rjl/ol = (1 + p)~ 2 by means of optimal noise 
shaping. For p = 0.85 the obtainable improvements in weighted signal- 
to-noise ratio are 5.6 dB and 2.4 dB, respectively. It is interesting to note 
that a noise-shaping improvement of about 1 dB has been calculated in 
Refs. 25 and 31 on the basis of experimentally determined speech spectra 
and noise-weighting functions. The total improvement is 8.0 dB for 
D*PCM or NFC, and it is 6.5 dB for prefiltered DPCM. 

The achievable error variances of various coding schemes can be de- 
termined by using eqs. (60) and (64)-(73): 



PCM: min {<xf\ = e 2 - . ... a* (79a) 



1+p 2 

(1+P) 2 
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Noise-feedback PCM: min {of} = t\ - — a\ (79b) 

(1 + pr 

D*PCM and NFC: min \af\ = e% o\ (79c) 

1 +P 

DPCM: min \<jf\ = e 2 { - " f )(1 7 p2) <r 2 (79d) 

(1 + p) 

(2\ 2 
-) <d (79e) 

Television signals. An average video spectrum is flat for frequencies 
below the line rate and falls at about 6 dB per octave through the rest 
of the band. A weighting function for such a signal has a negative slope 
of about 3 dB per octave. 7,31,32 We assume a weighting 



S Ju) = c 2 VSAa) (80) 

and it is not difficult to show that the reduction factor for noise shaping 



is 



ll = = = (81) 

2F(-, p ) l + i p a + — p* + ... 
\2 / 4 6<T 

for this choice of the weighting function. Using p = 0.9625, we find a 
performance improvement of 11.3 dB obtainable by optimal prediction 
and a noise-shaping gain of 2.4 dB. This latter figure is in good agreement 
with the results of calculations based on experimental data. 25,31 

3.4 Absolute performance bounds 

In the foregoing sections we have optimized various coding schemes 
whose structures had been given beforehand. It is useful to compare the 
performance of this class of encoders-decoders with absolute perfor- 
mance bounds by consulting the distortion-rate function. 12 A detailed 
discussion of these bounds for speech and television signals has been 
given by O'Neal. 31 In the following we shall restrict our attention to 
source coding of stationary ergodic Gaussian processes. For a fixed rate 
R there exists a minimum possible average distortion D which is a lower 
bound for any coding scheme. As above we adopt a frequency-weighted 
mean-squared error as our distortion measure. D and R are then related 
parametrically as follows: 33,34 

D{<p) = — f min{<?, S w (u) • S x (a>))cfco (82) 

R\<f>) = — I max 0, log 2 do). (83) 

Air J-* [ ^ 
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Next we define the weighted error spectrum by 

D=— f* S t (u)d<o (84) 

2ir %)~rc 

and, by comparing it with eq. (82), we find 

*«-{* i»<aUM-4M (86) 

I Su, (co)S x (co) otherwise. 

Over the frequency range where ip > S w (a>)-S x (ci)) we have S t (u>) = 
S w (u)'S x ((j)). This implies that no signal has to be transmitted over this 
frequency range since such a measure produces just this error spectrum. 
Over the frequency range where <p^S w (u))- S x {u>) we have S t {u>) = (p, i.e., 
the weighted error spectrum must be constant. Section 3.2 has shown 
that this requirement is met by the NFC scheme. In this latter case of 
small distortions we have D(<p) = <p = D and, by combining eqs. (22), (49), 
and (83), we obtain 

D = 2-™' V 2 x -vl- (86) 

A comparison with eq. (60) indicates that optimal NFC coding is very 
close to the distortion-rate bound D. The difference is in the first 
right-hand term of these equations because t\ > 2~ 2R for single-letter 
quantizers (see Table IV). We finally note that the three right-hand 
terms in eq. (86) correspond to the terms Tq, Tp, and T§ in O'Neal's 
paper. 31 

IV. TRANSMISSION ERRORS 

Noise-feedback coding schemes (including D*PCM and DPCM) are 
affected differently from PCM systems by bit errors on the communi- 
cation channel because the decoder loop causes an error propagation 
while a PCM error does not propagate in time. The objective of this sec- 
tion is to show the effects of transmission errors in predictive coding 
systems using some of the results of our above analysis. We shall con- 
centrate on two coding schemes, D*PCM and DPCM, and we shall only 
use the unweighted mean-squared error criterion. Recall that DPCM and 
NFC are identical in this case. 

4.1 PCM 

Let us assume that quantizing noise q n and channel noise c n can be 
modelled as additive noise sources. Thus the total error is 

t n =qn+C n (87) 

and its variance is 

af=a 2 q +^+2E[q n -c n ]. (88) 
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Table IV — Quantization error variances e 2 q (Max-quantizers) and 
channel coefficients 7 





1 


,2 


7 




1 bit 


2 bit 


lbit 


2 bit 


Uniform pdf 
Gaussian pdf 
Laplacian pdf 
Gamma pdf 


0.25 
0.363 
0.5 
0.667 


0.0063 
0.118 
0.176 
0.232 


3.0 
2.55 
2.0 
1.33 


3.75 
4.65 
5.3 
6.28 



Totty and Clark have shown 35 that channel errors and quantization 
errors are uncorrelated if the quantizer structure is that of Max. 36 These 
quantizers minimize the variance of the quantization noise but not 
necessarily that of the total error. This approach is of interest if a coding 
scheme has to operate on noisy channels with small bit-error probabil- 
ities which additionally are unknown or changing.* It is also justified by 
our observation that the step-size of quantizers with a low number of 
levels is not critical. The channel error variance depends on the bit-error 
probability P, on the density function of the signal being quantized, and 
on a\ (because the input variance determines the quantizer step-size 
scaling). Thus we have 

mini<r c 2 ) = Z-vl (89) 

PCM 

and the normalized channel error variance can be written as 

t 2 c = y-P (90) 

provided that the codewords are only affected by single bit errors. The 
channel coefficients 7 can be derived following an approach in Ref. 39. 
Table IV lists these values for 1-bit and 2-bit quantizers. 19 In the case 
of 1 bit, the channel coefficient is simply given as 

7 = 4.(1-4). (91) 

In Table IV the 7-values for 2-bit quantizers are given for the folded 
binary code with the exception of the uniform probability density 
function whose 7 -value is lower in the case of a natural binary code and 
is given by: 40 

6 c 2 =4.P(l-e5) (92) 

where 

e 2 q = 2-2B (93) 

with B as the number of bits. 



f A re-optimization of quantizers for noisy channels has been discussed in Refs. 37 and 
38. 
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The total error variance can thus be calculated as 

min {«r?} = (4 + 6 C 2 ) • <% = (* 2 + 7 • P) • a\ (94) 

PCM 

on the assumption of a vanishing correlation between the two errors. 

4.2 Noise-feedback coding 

The analysis of the predictive quantizing systems in the presence of 
channel errors is essentially identical to that of the last section if we 
substitute the nonwhite noise source S n (w) in eq. (47) with 

S„(a>) = <r 2 |l-Q(a>)| 2 +<x 2 

= [ £ 2 |l-Q(a;)|2 + € 2 ].^ (95) 

and if the assumption of eq. (54) still holds. Thus it is possible to reop- 
timize the various coders by following the same procedure as in the last 
section. We shall concentrate in this section on two coding schemes, 
D*PCM and DPCM. The objective here is to show that the D*PCM per- 
formance provides a bound for all predictive quantizing schemes if the 
channel is noisy. 

For both schemes the contribution of the channel errors of on the total 
error variance can directly be derived from the D*PCM results of Section 

2.3 by replacing e 2 , with € 2 . We then have, from eq. (29), 

D*PCM: a c = €c- a- da* 
DPCM: a 2 = e 2 • a ■ o-j (96) 

where a is the power transfer factor of eq. (27). Therefore the variance 
of white noise on the channel is increased in the decoder network by a 
factor a > 1. This error accumulation does not imply that the effect of 
transmission errors in D*PCM and DPCM is more severe than in PCM, 
because the generated noise variance depends now on the variance of 
the difference signal and can thus be influenced by the prefilter. Gains 
over PCM even for noisy channels have indeed been reported recently. 41,42 
The discussion of Section 2.3 has already shown that the total MSE can 
be smaller than that of PCM; it has also shown that coder and decoder 
should have inverse networks. 

4.2.1 D'PCM 

Transmission errors contribute to the total error in exactly the same 
way as quantization noise. The total error 

tn = (q n + c n )*h n (97) 

has a variance 

«,?» (4+ €?)•«.<&. (98) 
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An optimized D*PCM scheme minimizes at the same time both error 
contributions. The mean-squared errors are those given in Section 2.3 
if e 2 is replaced with cjj + t 2 . Note that the optimal filters in D*PCM 
schemes do not depend on the bit-error rate. We have seen that half- 
whitening of the input spectrum provides a D*PCM performance bound. 
From eq. (36) we conclude that the minimum channel error variance is 
given as 

min \a 2 c \ = e 2 c a 4 vi. (99) 

D'PCM 

4.2.2 DPCM 

Section 3.2 has revealed that D*PCM has the same total quantization 
error variance as PCM if the input is full-whitened (MMSE prediction). 
The D*PCM postfilter is then identical with that of DPCM if this latter 
scheme has been optimized for the noiseless channel. These observations 
imply that transmission errors cause the same channel error variances 
in DPCM as in PCM if an MMSE predictor is employed. Smaller MSE 
values, i.e., improvements over PCM, can be gained by reducing the 
whitening effect. The quantization noise MSE, however, increases then. 
The total error 

tn = Qn + C„ * k n (100) 

has a variance 

of = (e 2 + ae 2 )a 2 d . (101) 

In the case of high bit-error probabilities (ae 2 » t 2 -) the total MSE is 
minimized if the prediction network performance is close to that of an 
optimized D*PCM coder and eq. (99) provides a lower bound in channel 
error variance for DPCM (and NFC) in the case of noisy channels. For low 
bit-error rates the best predictor will have a characteristic between 
full-whitening (error-free transmission) and half-whitening (D*PCM 
bound for noisy channels). Therefore 

min \(t 2 \ > min \a 2 \ > min [<%} (102) 

PCM DPCM D*PCM 

Example 6: Assume a previous-sample 2-bit DPCM and a Gaussian (not 
necessarily Markovian) source with adjacent-sample correlation p = 
0.85. Let the bit-error probability be P = 0.05. From eq. (90) and Table 
IV we have e, = 0.118 and e 2 = 0.233. Figure 7 shows the dependence of 
the signal-to-noise ratio of this DPCM scheme on the value a of its pre- 
dictor coefficient both for the noiseless and the noisy channel. The 
theoretical results have been calculated from eq. (101) using eqs. (16) 
and (31). 
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Fig. 7 — DPCM performance on noisy channels. Two-bit quantization of Gaussian source 
with adjacent-sample correlation p = 0.85. Folded binary code with bit-error rates P = 
and P = 0.05. 

It is seen that DPCM performs better than PCM if the predictor is ap- 
propriately chosen. The differences between theory and measurements 
for low values of the predictor coefficient are again a consequence of the 
fact that the quantization error has been assumed to be not correlated 
with the input signal. For quantizers with a low number of levels this 
assumption only holds if the signal being quantized is uncorrelated, i.e. 
for predictor coefficients close to p. Notice that the crosspoint between 
PCM and DPCM performance is reached for a value of a which is slightly 
higher than p; this deviation from the predicted performance is a con- 
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Fig. 8 — Noiseless channel feedback dpcm. 



sequence of the noise feedback. Figure 7 also compares the DPCM per- 
formance in the case of a noisy channel with the equivalent D*PCM 
performance which has been obtained from eqs. (98), (14), and (31). It 
is apparent that this one-tap D*PCM performance is a very useful bound 
of the DPCM performance. The optimal value of the DPCM predictor 
coefficient is also close to the optimal D*PCM coefficient a opt as given 
in eq. (33) (a opt ~ 0.57 for high bit-error probabilities). The choice of a opt 
in accordance with eq. (33) for very noisy channels has first been men- 
tioned in Ref. 41. 



4.2.3 Noiseless channel feedback DPCM 

In D*PCM the predictors of coder and decoder operate on slightly 
different signals, because there are quantizing noise and channel noise, 
respectively, in the intervening path. In DPCM both predictors operate 
on the same signal, viz. the sequence of reconstructed samples, if only 
the channel is noiseless. In the case of channel errors the predictions are 
again different, and the channel noise has the same effect on the overall 
MSE as in D*PCM. Let us assume that a noiseless channel from decoder 
to coder is available (see Fig. 8). It is then possible to ensure identical 
predictions via the feedback channel by retransmitting the prediction 
values calculated at the decoder. Notice that this scheme is identical to 
standard DPCM if the transmission of the encoded difference samples 
to the decoder is noiseless. In the case of channel errors, however, we have 
a total error 



tn = Qn + Cn 



(103) 



of variance 



*? = <*; + €?).<* 



(104) 



The contrast to D*PCM and standard DPCM is quite clear; the feedback 
is now around quantizer plus noisy channel, and thus error accumulation 
in the decoder loop has been totally avoided. 
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Example 7: In the case of previous-sample prediction the results of 
Example 1 (DPCM with analysis of the effects of noise feedback) apply. 
The optimum choice of the predictor-coefficient is given by eq. (17) if 
ejj is replaced with t 2 q + e%. To demonstrate the improvements let us 
use the figures of the previous example. We have e% + e* = 0.351, and 
hence a opt = 0.755. We finally find [from eqs. (16) and (104)] a signal- 
to-noise ratio of 9.0 dB instead of 7.7 dB in the case of DPCM without 
noiseless channel feedback. 



V. SUMMARY 

There is a great interest in low bit-rate transmission of speech and 
television signals. Especially for acoustic signals it is well known that 
the subjective performance of a coder is strongly affected by the way in 
which quantizing distortion is distributed in frequency. In this paper 
we have compared coding schemes which employ prediction to exploit 
the inherent redundancy of these signals and which employ noise- 
shaping for optimizing the subjective performance on the basis of a 
frequency-weighted error criterion. First we have shown that DPCM 
outperforms D*PCM (a predictive scheme that lacks the feedback around 
the quantizer) for all nonwhite input spectra if the performance criterion 
is the unweighted total error variance. We have then used a noise-feed- 
back coding structure as a framework for a unified analysis of predictive 
quantizing schemes. With this structure a minimum frequency-weighted 
error variance can be obtained by a proper shaping of the signal spectrum 
and the quantization noise prior to transmission. A comparison of this 
error variance with the distortion bound as given by the distortion-rate 
function for Gaussian signals has revealed that the performance of the 
noise-feedback coder is almost optimal for this class of signals. We have 
also shown that this coding structure degenerates to DPCM in the case 
of unweighted noise, and to D*PCM if the weighting function is inverse 
to the input spectrum. The performance results for these optimal coders 
have then been compared with those of suboptimal schemes including 
noise-feedback PCM and DPCM with prefiltering. For first-order Markov 
sources which often serve as a model of actual input spectra we have been 
able to derive simple explicit results in terms of the autocorrelation 
coefficient. In the last part we have examined the effects of channel 
transmission errors on the overall performance of these predictive 
quantizing schemes. We have shown that these coders when appro- 
priately designed are less sensitive to channel errors than PCM. In D*PCM 
channel errors contribute to the total error in exactly the same way as 
quantization noise. Thus the D*PCM results provide a guideline for the 
optimization and a bound for the performance of DPCM. 
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